Medical image rendering method and apparatus

ABSTRACT

A medical image processing apparatus comprises processing circuitry configured to: obtain volumetric medical imaging data comprising a voxel value; obtain an opacity value corresponding to the voxel value; obtain an extinction color and/or transmission color corresponding to the voxel value; modify the extinction color and/or transmission color using the opacity value, wherein the modifying of the extinction color and/or transmission color is performed using a combined opacity model that combines a first opacity model and a second, different opacity model, such that the first opacity model makes a higher contribution to the modifying than the second opacity model at lower values of opacity, and the second opacity model makes a higher contribution to the modifying than the first opacity model at higher values of opacity; and render the volumetric medical imaging data using the modified extinction color and/or transmission color.

FIELD

Embodiments relate generally to a method of image rendering, for examplea method of image rendering in which a master opacity function is usedto modify extinction colors and/or transmission colors used in globalillumination rendering.

BACKGROUND

It is known to render images from volumetric imaging data, for examplefrom volumetric medical imaging data. A set of volumetric imaging datamay be referred to as an image volume. The set of volumetric imagingdata may comprise a plurality of voxels with associated voxel values,with each voxel being representative of a corresponding spatial locationin a medical imaging scan. For example, in the case of computertomography (CT) data, the voxel value associated with each voxel may bea voxel intensity value that is representative of an attenuation ofapplied X-ray radiation at the location represented by the voxel.

Direct Volume Rendering (DVR) is a method for volume rendering. DVR maybe considered to be an industry standard for volume rendering.

DVR renders a volumetric imaging data set by modelling a cloud ofemissive material. Each point in a volume represented by the volumetricimaging data set is assigned properties of color and opacity. The colorand opacity may be expressed as RGBA values, where R is red, G is green,B is blue and A is alpha. Typically, alpha (which may be written as A, aor α) is a measure of opacity in which a value of 0% indicatestransparency and a value of 100% indicates full opacity.

In known DVR systems, a user may select rendering properties using a DVRmodel that comprises a set of color keysand an opacity curve.

A color key may be a color control point that may be used by a user toinput or adjust a color value for a given intensity value. A set ofcolor keys may be defined on a color curve. The color keys may be usedto adjust the color curve. The color curve may be a plot of coloragainst intensity value.

Color values (for example, color keys on a color curve) may be convertedinto tables of color values. A table may comprise a respective colorvalue for each of a plurality of intensity values) by usinginterpolation and/or extrapolation. Interpolation may be used tointerpolate color values between color keys. The common method forinterpolation is linear, but cubic or higher order polynomials may beused. Extrapolation may be used to extrapolate color values before thefirst color key and after the last color key. A common extrapolationmethod is clamping to the nearest value.

The opacity curve may be a plot of opacity value versus voxel value,where voxel values are intensity values. Typically, low intensity valuesmay be mapped to low opacity (high transparency), such that voxels thatare representative of material having low attenuation (for example,voxels that are representative of air) are rendered as entirely ormostly transparent. High intensity values may be mapped to high opacityvalues, such that voxels that are representative of material having highattenuation (for example, voxels that are representative of bone) arerendered as entirely or mostly opaque. However, different mappings mayalso be made, for example to emphasize or de-emphasize particular tissuetypes in a resulting rendered image.

The term preset may be used to refer to a mapping of color values andopacity values to intensity values. For example, a preset may comprisean opacity curve as described above, and a transfer function that mapscolor values (for example, red, green and blue values) to intensityvalues.

In DVR, opacity is not dependent on color. A single opacity value ismapped to each intensity value. The single opacity value is used for therendering of all color channels.

Global illumination (GI) is another type of volume rendering. Globalillumination rendering has more degrees of freedom than DVR. In globalillumination, a lighting model may be used that includes both directillumination by light coming directly from a light source and indirectillumination, for example illumination by light that has been scatteredfrom another surface.

In some global illumination rendering methods, an image is renderingfrom a volumetric imaging data set using a two-pass method in which afirst pass creates a light volume, and a second pass uses the lightvolume to render an image for display.

The first pass may comprise a traversal from the light source into thevolumetric imaging data set, in which virtual light is cast into thevolumetric imaging data set. The irradiance due to the light source maybe determined at each of a large array of points in the volumetric imagedata set using absorptive properties assigned to the voxels independence on the voxel intensities. The irradiance values at the arrayof points may be stored as a light volume. The light volume may bestored in memory. The light volume may be independent of the viewpoint.

A second pass may comprise a traversal through the light volume from acamera, using the light volume to provide global lighting information.Rays may be cast from the camera (for example, one ray for each pixel ofthe resulting rendered image), and irradiances from points along eachray may be integrated to provide pixel color values for a final renderedimage.

In GI, each point in the volumetric imaging data set is assigned aproperty that may be referred to as an extinction color, absorptioncolor, or attenuation color. In the description below we use the termextinction color. The extinction color may be expressed as an RGB color.

The extinction color for a point in the volume is representative of thecolor-dependent absorption of light at that point in the volume. Theextinction color may be considered to be representative of the amount ofenergy removed for each channel at each point.

For example, consider a voxel having an extinction color C_(extinction)of r, g, b={0.3, 0.4, 0.7}. When virtual light passes through thisvoxel, 30% of the energy in the red channel is removed; 40% of energy inthe green channel is removed; and 70% of the energy in the blue channelis removed.

We consider a change in irradiance between sampling points along a rayusing the extinction color.I _(red)(i+1)=I _(red)(i)−C _(extinction_red) I _(red)(i)=0.7I _(red)(i)

where I_(red)(i) is the irradiance in the red channel at voxel i andI_(red)(i+1) is the irradiance in the red channel at voxel i+1, andC_(extinction_red) is the red component of the extinction colourC_(extinction).

Similarly,I _(green)(i+1)=I _(green)(i)−C _(extinction_green) I _(green)(i)=0.6I_(green)(i)I _(blue)(i+1)=I _(blue)(i)−C _(extinction_blue) I _(blue)(i)=0.3I_(blue)(i)

A transmission color C_(transmission) may also be defined. Thetransmission color represents the amount of energy remaining in eachchannel, whereas the extinction color represents an amount of energythat is removed from each channel. The transmission color is thefraction of light that manages to pass through the material.C _(transmission_red)=1−C _(extinction_red)C _(transmission_green)=1−C _(extinction_green)C _(transmission_blue);=1−C _(extinction_blue)

Consider the example above in which the extinction color C_(extinction)is r, g, b={0.3, 0.4, 0.7}. The transmission colour C_(transmission) isthen r, g, b={0.7, 0.6, 0.3}.

The change in irradiance between one sample and the next sample may alsobe written in terms of transmission color as, for example,I _(red)(i+1)=C _(transmission_red) I _(red)(i)=0.7I _(red)(i)I _(green)(i+1)=C _(transmission_green) I _(green)(i)=0.6I _(green)(i)I _(blue)(i+1)=C _(transmission_blue) I _(blue)(i)=0.3I _(blue)(i).

The use of extinction colors and/or transmission colors that arecolor-dependent may enable realistic tissue effects due tochroma-dependent attenuation. For example, white light passing throughhuman tissue may become redder as it passes through the tissue due tothe tissue preferentially absorbing blue and green components of thewhite light. The use of extinction colors may allow such reddening, andother tissue effects, to be rendered. For example, in the case above inwhich a voxel has an extinction color of r, g, b={0.3, 0.4, 0.7}, greenand blue are absorbed more strongly than red, and so light passingthrough that voxel will be reddened.

Rendering using a global illumination model is starting to becomeavailable to users. When the GI model is used correctly, the results maybe very good. However, it has been found that many users find itdifficult to create excellent images using GI. In particular, users mayfind it very difficult to create presets in GI.

Users may be used to using a single opacity curve in DVR rendering. Thereliance of GI on color-dependent optical density and/or color-dependentabsorption may mean that the use of a single (non-color-dependent)opacity curve may be considered not to be compatible with a GI model.Optical density describes energy loss through both absorption andscattering.

One option could be to allow users of GI models to define separateopacity curves for each color. However, it has been found that usersfind it extremely difficult to define separate opacity curves in thismanner. The different absorption of different colors may cause colors toappear in rendering in a way that was not anticipated by the user whenthe user defined the opacity curves. If the user wishes to make a changein the appearance of a rendered color, the user may not know whichchange should be made to the preset to obtain the desired change inappearance of the rendered color.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are now described, by way of non-limiting example, and areillustrated in the following figures, in which:

FIG. 1 is a schematic illustration of an apparatus in accordance with anembodiment;

FIG. 2 is a flow chart illustrating a rendering method in accordancewith an embodiment;

FIG. 3 is a plot of a mapping function; and

FIGS. 4a, 4b and 4c are plots of color against opacity and thickness fora deepening factor of 1, 1.5 and 2 respectively.

DETAILED DESCRIPTION

Certain embodiments provide a medical image processing apparatuscomprising processing circuitry configured to: obtain volumetric medicalimaging data comprising a voxel value; obtain an opacity valuecorresponding to the voxel value; obtain an extinction color and/ortransmission color corresponding to the voxel value; modify theextinction color and/or transmission color using the opacity value,wherein the modifying of the extinction color and/or transmission coloris performed using a combined opacity model that combines a firstopacity model and a second, different opacity model, such that the firstopacity model makes a higher contribution to the modifying than thesecond opacity model at lower values of opacity, and the second opacitymodel makes a higher contribution to the modifying than the firstopacity model at higher values of opacity; and render the volumetricmedical imaging data using the modified extinction color and/ortransmission color.

Certain embodiments provide a medical image rendering method comprising:obtaining volumetric medical imaging data comprising a voxel value;obtaining an opacity value corresponding to the voxel value; obtainingan extinction color and/or transmission color corresponding to the voxelvalue; modifying the extinction color and/or transmission color usingthe opacity value, wherein the modifying of the extinction color and/ortransmission color is performed using a combined opacity model thatcombines a first opacity model and a second, different opacity model,such that the first opacity model predominates at low values of opacityand the second opacity model predominates at high values of opacity; andrendering the volumetric medical imaging data using the modifiedextinction color and/or transmission color.

Certain embodiments provide a medical image processing apparatuscomprising processing circuitry configured to: obtain volumetric medicalimaging data comprising a voxel value; obtain an extinction color and/ortransmission color corresponding to the voxel value; obtain a value fora deepening factor that is representative of saturation; modify theextinction color and/or transmission color using the value for thedeepening factor; and render the volumetric medical imaging data usingthe modified extinction color and/or transmission color.

An image rendering apparatus 10 according to an embodiment isillustrated schematically in FIG. 1. The image rendering apparatus 10comprises a computing apparatus 12, which in this case is a personalcomputer (PC) or workstation. The computing apparatus 12 is connected toa CT scanner 14, a display screen 16 and an input device or devices 18,such as a computer keyboard and mouse.

In other embodiments, the CT scanner 14 may be supplemented or replacedby a scanner in any appropriate imaging modality, for example acone-beam CT scanner, MRI (magnetic resonance imaging) scanner, X-rayscanner, PET (position emission tomography) scanner, SPECT (singlephoton emission computed tomography) scanner, or ultrasound scanner.

In the present embodiment, sets of volumetric imaging data are obtainedby the CT scanner 14 and stored in data store 20. The image renderingapparatus 10 receives sets of volumetric imaging data from data store20.

In other embodiments, sets of image data may be obtained by any suitablescanner and stored in data store 20. In alternative embodiments, theimage rendering apparatus 10 receives sets of volumetric imaging datafrom a remote data store (not shown) which may form part of a PictureArchiving and Communication System (PACS). The image rendering apparatus10 may not be connected to a scanner 14.

Computing apparatus 12 provides a processing resource for automaticallyor semi-automatically processing volumetric imaging data sets. Computingapparatus 12 comprises a central processing unit (CPU) 22.

The computing apparatus 12 includes input circuitry 24 configured toreceive user input; modulation circuitry 26 configured to modifyextinction colors and/or transmission colors in accordance with opacity;and rendering circuitry 28 configured to render images from volumetricdata.

In the present embodiment, the circuitries 24, 26, 28 are eachimplemented in computing apparatus 12 by means of a computer programhaving computer-readable instructions that are executable to perform themethod of the embodiment. However, in other embodiments, the variouscircuitries may be implemented as one or more ASICs (applicationspecific integrated circuits) or FPGAs (field programmable gate arrays).

The computing apparatus 12 also includes a hard drive and othercomponents of a PC including RAM, ROM, a data bus, an operating systemincluding various device drivers, and hardware devices including agraphics card. Such components are not shown in FIG. 1 for clarity.

The apparatus of FIG. 1 is configured to perform a series of stages asillustrated in overview in the flow chart of FIG. 2.

FIG. 2 is a flow chart illustrating in overview a method of rendering inaccordance with an embodiment. In the embodiment of FIG. 2, the userselects a single opacity value for each material, for example by using asingle master opacity curve. The user's selection is then translatedinto opacity properties that are suitable for use in GI where opacity iscolor-dependent. The user may find the use of a single master opacitycurve to be intuitive. The user may be familiar with the use of a singlemaster opacity curve in DVR rendering.

In the below embodiments, for simplicity we consider the user'sselection of properties (opacity, transmission color and/or extinctioncolor, and optionally deepening factor) for a range of intensity valuesthat are representative of a single material. The user selects only oneopacity and one transmission color and/or extinction color (andoptionally one deepening factor).

However, in practice the user may select different properties formultiple different materials, for example different tissue types. Insome embodiments, for a given material, the user selects properties thatvary with intensity within a range of intensities that arerepresentative of that material.

Turning to FIG. 2, at stage 30, the user selects an opacity value α fora material, for example soft tissue. The opacity value may be a valuefrom 0% (fully transparent) to 100% (fully opaque). The user may use anysuitable input device and input method to select the opacity value. Theinput circuitry 24 receives an input which comprises or represents theopacity value selected by the user.

In alternative embodiments, the user may select a transparency value T,where transparency is related to opacity by T=1−α. In the descriptionbelow, references to opacity value may be applied to transparency valueand vice versa, if a suitable conversion between opacity value andtransparency value is made.

In other embodiments, the user may select opacity values for anysuitable intensity values or ranges of intensity values. The user maydefine an opacity curve that relates opacity to intensity value over arange of intensity values. The opacity curve may be referred to as amaster opacity curve, master opacity function, or master opacity ramp.

In further embodiments, the input circuitry 24 uses one or more storedopacity values instead of receiving user input. The stored opacityvalues may have been predetermined using any suitable method.

As discussed above, GI does not use opacity in the same way as DVR.However, the user may expect that a high opacity input will result in amore opaque appearance of the material in a rendered image, and a lowopacity input will result in a more transparent appearance of thematerial in a rendered image.

At stage 32, the user selects a transmission color C_(transmission). Thetransmission color comprises red, green and blue components. Forexample, the transmission color may be r, g, b={0.7, 0.6, 0.4} asdescribed above.

The user may use any suitable input device and method to select thetransmission color. For example, the user may select the color on acolor picker, or input numerical values. The input circuitry 24 receivesan input which comprises or represents the transmission color selectedby the user.

In other embodiments, the user may select an extinction color instead ofa transmission color. As described above, a transmission color may beconverted into an extinction color and vice versa. In the descriptionbelow, references to transmission color may be applied to extinctioncolor and vice versa, if a suitable conversion between extinction colorand transmission color is made.

In the present embodiment, the user selects one transmission color forthe range of intensity values for which the opacity value was determinedat stage 30. In other embodiments, the user may select transmissioncolors and/or extinction colors for any suitable intensity values orranges of intensity values. The user may select transmission colorsand/or extinction colors for multiple materials, for example multipletissue types. The user may create, select, or modify a function, forexample a transfer function, which relates transmission color and/orextinction color to intensity value over a range of intensity values.

In further embodiments, the input circuitry 24 uses one or more storedtransmission colors and/or extinction colors instead of receiving userinput. The stored transmission colors and/or extinction colors may havebeen predetermined using any suitable method.

At stage 34, the modulation circuitry 26 modifies the transmission colorthat was selected at stage 32. In the present embodiment, the modulationcircuitry 26 modifies the transmission color using the opacity valuethat was selected at stage 30. The modulation circuitry modifies thetransmission color using a combination of two opacity models, which inthe present embodiment may also be referred to as opacity modulationmodels. The combination of the two opacity models may be referred to asa combined opacity model, or a combined opacity modulation model.

In the present embodiment, the two opacity models are an extinctionmodulation model and a transmission modulation model. The extinctionmodulation model modifies an extinction color by multiplying it by anopacity value. The transmission modulation model modifies a transmissioncolor by multiplying it by a transparency value.

In embodiments described below, the modifying of the transmission colorcomprises modulation. The term modulation is used to mean multiplicationby a single factor. In other embodiments, any suitable way of modifyingthe transmission color and/or extinction color may be used, which may ormay not comprise a modulation.

We express the modification of the transmission color using the combinedopacity model using the equation below.C′″ _(transmission) =A(α)C′ _(transmission) +B(α)C″ _(transmission)

where C′″_(transmission) is the transmission colour as modified by thecombined opacity model, C′_(transmission) is the transmission color asmodified by the extinction modulation model, C″_(transmission) is thetransmission colour as modified by the transmission modulation model,and A(α) and B(α) are linear functions of opacity.

In other embodiments, any suitable functions may be used for A(α) andB(α). In some embodiments, the functions may not be linear.

The extinction modulation model modulates an extinction color bymultiplying it by an opacity value.C′ _(extinction) =αC _(extinction)

Written in terms of transmission color,

C_(transmission)^(′) = 1 − C_(extinction)^(′) = 1 − α C_(extinction) = 1 − α(1 − C_(transmission)) = 1 − α + α C_(transmission)

The transmission modulation model modulates a transmission color bymultiplying it by a transparency value.C″ _(transmission) =TC _(transmission)

Written in terms of opacity,C″ _(transmission)=(1−α)C _(transmission)

In the present embodiment, the extinction modulation model andtransmission modulation model are combined as a linear combination. Theextinction modulation model and transmission modulation model arecombined such that if the opacity value is 0, then only the extinctionmodulation model is applied. If the opacity value is 100%, only thetransmission modulation model is applied. For values of opacity between0% and 100%, a linear combination of the opacity models is applied. Inthe present embodiment, this is achieved by setting A(α)=(1−α) andB(α)=α.

Expanding out and substituting the equations above gives us:

$\begin{matrix}{C_{transmission}^{\prime\prime\prime} = {{{A(\alpha)}C_{transmission}^{\prime}} + {{B(\alpha)}C_{transmission}^{''}}}} \\{= {{\left( {1 - \alpha} \right)\left( {1 - \alpha + {\alpha\; C_{transmission}}} \right)} +}} \\{\alpha\left( {C_{transmission} - {\alpha\; C_{transmission}}} \right)} \\{= {{\left( {{{- 2}\;\alpha^{2}} + {2\;\alpha}} \right)C_{transmission}} + \alpha^{2} - {2\;\alpha} + 1}}\end{matrix}$

A linear mapping function between the two linear models produces aquadratic master opacity mapping. The function relating the originaltransmission color C_(transmission) that is input by the user and themodified transmission color C′″_(transmission) is a quadratic functionof opacity.

FIG. 3 shows a plot of the mapping function of C′″_(transmission). Thevertical axis, z, of FIG. 3 shows transmission of red as a function of aselected red amount, r, and opacity, which is indicated by a in FIG. 3.The selected red amount is the amount of red in the transmission colorselected by the user at stage 32.

The combined opacity model is selected such that the modifying of thetransmission color by the opacity value has certain properties that areexpected by the user.

In particular, if the user sets the opacity value to 0, the user willexpect the material to be rendered as transparent. In the combinedopacity model above, setting the opacity value to 0 sets thetransmission color to 1, i.e. fully transparent.

If the user sets the opacity value to 100%, the user will expect thematerial to be rendered as opaque. In the combined opacity model above,setting the opacity value to 1 sets the transmission color to 0, i.e.fully opaque.

Using either the extinction modulation model or transmission modulationmodel alone would not provide an opacity model having the propertiesexpected by a user. This point will be discussed in more detail below.

In other embodiments, a different combined opacity model may be used.The combined opacity model may be a combination of any two or moreopacity models, for example any linear combination of two or moreopacity models, such that the first opacity model has a highercontribution to the modifying of the transmission color than the secondopacity model at lower values of opacity (for example, values of opacitynear zero), and the second opacity model has a higher contribution tothe modifying of the transmission color that the first opacity model athigher values of opacity (for example, values of opacity near 100%).

The combined opacity model may be any model that is smooth andcontinuous. In some embodiments, the combined opacity model may be anymodel that has the property of providing a fully opaque material at 100%opacity and a fully transparent material at 0% opacity.

A first opacity model and a second opacity model may be defined suchthat if the opacity value is zero, the transmission color is modifiedusing the first opacity model. If the opacity value is 100%, thetransmission color is modified using the second opacity model. Forvalues between 0% opacity and 100% opacity, the transmission color ismodified using a linear combination of the first opacity model and thesecond opacity model.

In further embodiments, any analytical interpolation function may beused to interpolate between the first opacity model and the secondopacity model. The interpolation function may comprise, for example, acurve or a piecewise analytical function. In some embodiments, aninterpolation function is implemented by using a lookup table tointerpolate between the models.

In some embodiments, first, second and third ranges of intensity valuesare defined. In a first, lower range of intensity values, only the firstopacity model is used. In a second, higher range of intensity values,only the second opacity model is used. In a third range of intensityvalues that is intermediate between the first range and second range, acombination of the first opacity model and second opacity model is used,for example a linear combination.

At the end of stage 34, the modulation circuitry 26 outputs the modifiedtransmission color C′″_(transmission).

At stage 36, the rendering circuitry 28 renders an image from thevolumetric imaging data set using the modified transmission color thatwas output at stage 34. In the present embodiment, the renderingcomprises global illumination rendering. In other embodiments, therendering may comprise any physical rendering method in which acolor-dependent absorption and/or optical density is used. A physicalrendering method may be a rendering method that attempts to emulatereal-world optical behavior.

By modifying the transmission color using the combined opacity model asdescribed above, a user choice of a single opacity value may be appliedto a rendering model in which opacity varies by color.

The modified transmission color may be used in an irradiance calculationas shown below:

I_(red)(i + 1) = C_(transmission_red)^(′′′)I_(red)(i) = ((−2 α² + 2 α)C_(transmission) + α² − 2 α + 1)I_(red)(i)

with corresponding equations for I_(green) and I_(blue).

In the present embodiment, the same modified transmission color is usedfor all instances of the same material. In other embodiments, differenttransmission colors and/or different modifications may be used fordifferent regions. For example, tissue representing different organs maybe segmented, and different colors may be used in rendering thedifferent organs.

At stage 38, the user views the image that was rendered at stage 36, anddecides whether the rendered color of the material for which the userhad selected the transmission color is as good as the user had expected.If the user determines that the rendered color is good, the method ofFIG. 2 proceeds to stage 42. The user may decide that the rendered coloris good if the material is rendered with color properties (for example,hue and saturation) that were desired by the user.

If the user determines that the rendered color is not good enough, thanthe method of FIG. 2 proceeds to stage 40.

At stage 40, the user decides whether, in the user's opinion, the colorof the material in the rendered image lacks volumetric saturation.Volumetric saturation may be described as a deepness or richness ofcolor. The user may wish the color of the material to appear moresaturated.

If the user decides that the color of the rendered material does notlack volumetric saturation, the process of FIG. 2 proceeds to stage 44.

At stage 44, the user decides whether, in the user's opinion, the colorof the rendered material does not look good enough because it has anundesirable surface color. If the user considers the surface color to beundesirable, the process of FIG. 2 returns to stage 32, and the userselects a new transmission color. Stages 34 and 36 are repeated usingthe original opacity value and the newly-selected transmission color tocreate a new rendered image.

If the user determines that the surface color is not undesirable, andhas already determined at stage 40 that the material does not lackvolumetric saturation, then the process of FIG. 2 returns to stage 30and the user selects a new value for opacity.

Stages 34 and 36 are repeated using the newly-selected opacity value andthe original transmission color to create a new rendered image.

We now consider what happens at stage 40 if the user decides that, inthe user's opinion, the rendered material lacks volumetric saturation.If the user decides that the material lacks volumetric saturation, theprocess of FIG. 2 proceeds to stage 42. At stage 42, the user is giventhe opportunity to change a deepening factor. The deepening factor is apositive numerical value, which may also be referred to as a volumetricdeepening factor.

In some embodiments, the user interface displayed to the user comprisesa saturation dial. The saturation dial may be attached to each colorcontrol point (for example, a color key on a color curve), or attachedwhen the mouse is near a color control point. The user may control thedeepening factor by operating the saturation dial.

In the present embodiment, the deepening factor is set to 1 by default.If the user increases the deepening factor above 1, the color becomesmore saturated.

In other embodiments, the user may decide that the material color hastoo much volumetric saturation. The user may choose to decrease thedeepening factor such that the color becomes less saturated. Forexample, if the user has already applied a deepening factor in aprevious iteration of the method of FIG. 2, the user may decide toreduce the deepening factor applied.

The deepening factor may be implemented using any suitable method. Inthe present embodiment, the deepening factor is applied as an adjustmentto the modification of the transmission color that was made using thecombined opacity model.

In the present embodiment, the deepening factor d provides a furthermodulator of the color that is not directly related to the opacity. Inother embodiments, the deepening factor d may be used to modify thecolor in any appropriate manner.

In the present embodiment,C″″ _(transmission)=min((−2α²+2α)dC _(transmission)+α²−2α+1,1)

where C″″_(transmission) is the transmission colour as modified by boththe combined opacity model and the deepening factor, and d is thedeepening factor.

The deepening factor allows the user to saturate the color of tissuewithout producing oversaturated colors (colors with color channel valuesabove 1) which could not possibly be represented by the user interface.

FIGS. 4a, 4b and 4c are plots of color against opacity and thickness ofmaterial. FIG. 4a shows colors for a deepening factor of 1. FIG. 4bshows colors for a deepening factor of 1.5. FIG. 4c shows colors for adeepening factor of 2. Although the colors are intended to show as red,the plots have been re-rendered in greyscale. Saturated red shows as amedium grey. We note that in FIGS. 4a, 4b and 4c , opacity is shown on ascale from 0 to 255, where 255 is representative of 100% opacity.

Where the deepening factor is increased, the band of opacity values forwhich a saturated red color is produced is increase, and the red colorin that band intensifies.

In the present embodiment, the user changes a deepening factor to changethe appearance of a material in the rendered image. In otherembodiments, the user may use the deepening factor to change theappearance of any number of materials. The user may select which partsof a transfer function have increased volumetric saturation.

In some embodiments, the user may choose to apply a deepening factor tocertain regions of an images, for example to certain anatomical featuressuch as organs.

In other embodiments, the deepening factor may be applied automatically.In some embodiments, the deepening factor is used to indicate additionalinformation in an image.

In some embodiments, the deepening factor is used as a degree of freedomin image fusion. The deepening factor may be defined spatially and usedfor fusing a secondary volume. The secondary volume may comprise dataobtained using any suitable modality, for example CT perfusion, PET,SPECT, ultrasound Doppler, ultrasound electrograph, simulated fluiddynamics.

The modulation circuitry 26 may apply a deepening factor in dependenceon PET uptake or other functional parameters. For example, the deepeningfactor may be used to deepen the colors of the tissue where there is PETuptake. A respective value for the deepening factor for each of aplurality of positions in the image volume may be determined based on aPET uptake at each of those positions.

In other embodiments, the deepening of the color may be representativeof any suitable quantity, for example, a pressure, a velocity or avorticity.

In the method of FIG. 2, the deepening factor is applied in combinationwith modifying of the transmission color using the combined opacitymodel. In other embodiments, the deepening factor may be applied withoutany modifying of the transmission color or extinction color.

Stages of FIG. 2 may be repeated until the user decides at stage 38 thatthe material color is good. Once the user decides that the materialcolor is good, the method of FIG. 2 proceeds to stage 42. At stage 42,the user may interact with the rendered image. The user may navigatearound the rendered image, for example by changing a viewing orientationand/or a level of zoom. The user may add or move one or more clipplanes. The user may identify anatomical features in the rendered image,or obtain other medical information. The user may perform additionalprocesses on the rendered image, for example segmentation.

The combined opacity model uses a combination of the extinctionmodulation model and the transmission modulation model to produceresults at each end of the opacity spectrum (0% opacity and 100%opacity) which may be similar to the results that are expected by auser. The extinction modulation model is applied at the low opacity endof the opacity spectrum, and the transmission modulation model isapplied at the high opacity end of the opacity spectrum. The extinctionmodulation model makes a higher contribution to the modification of thetransmission color at lower values of opacity, and the transmissionmodulation model makes a higher contribution to the modification of thetransmission color at higher values of opacity.

By using the method of FIG. 2, a user interface may be provided in whichall colors are represented in the user interface in a way that isreasonably easy for the user to understand. A user may find thatchanging an opacity value produces a change in a rendered image that isas expected by the user.

The use of the combined opacity model may allow the use of a single setof color keys. The use of a single set of color keys may be verydesirable. The use of a single set of color keys may make the renderingeasier and more intuitive for a user. The user may be able to adjustcolor of a rendered image in a way that provides expected and intuitiveresults.

A user may be used to using a DVR model for rendering. The masteropacity curve that is typically used in DVR does not relate directly tothe chromatic optical density values that are used in other methods suchas GI, because the DVR model does not provide an accurate physicalrepresentation. However, in some circumstances the use of a singleopacity curve may be important to a user's understanding of an opticalmodel.

The master opacity ramp may be considered to be a cornerstone of apreset user interface. The use by the user of a single opacity curve maybridge a gap in understanding between the DVR model that the user isexperienced in using, and a new GI model. Keeping a model as close aspossible to the DVR model from the user's perspective may ease atransition between the technologies. By providing a user with a familiarway to manage color and opacity, the user's ability to create presetsmay be enhanced.

In the method of FIG. 2, the master opacity ramp is kept in a way thatmakes the model behave as expected. It may be expected that 0 opacityshould not attenuate the light at all. It may be expected that fullopacity should attenuate all of the light, because full opacity isrepresentative of infinite optical density. A principle of the methodused may be that colors picked by the user should be preserved as far aspossible. A deepening factor may be used to enhance a volumetric effectin places.

In the combined opacity model, a mapping function is created between thetwo models (extinction modulation model and transmission modulationmodel) so that the system acts like the extinction modulation model nearopacity=0 and smoothly transitions to the transmission modulation modelas opacity reaches full opacity. When opacity is low, the combinedopacity model behaves like the extinction modulation model. When opacityis high, the combined opacity model behaves like the transmissionmodulation model.

The combination of the extinction modulation model and transmissionmodulation model has properties that may be preferable to the propertiesof either the extinction modulation model alone, or the transmissionmodulation model alone.

We consider what would happen if the extinction modulation model alonewere to be used to modify the transmission color, i.e. if only theextinction modulation model were used across the full range of opacityvalues from 0% to 100% opacity.

As described above, in the extinction modulation model the extinctioncolor is modified by multiplying by opacity.C′ _(extinction) =αC _(extinction)

where C_(extinction) is the extinction colour modified using theextinction modulation model, and a is opacity.

We consider what happens in the extreme cases of opacity=0% andopacity=100%. As an example, we again consider the case in whichC_(extinction)={0.3, 0.4, 0.7} and we consider the example of the redchannel.

We consider a change in irradiance between sampling points along a rayusing the modified extinction color that is modified using theextinction modulation model.

I_(red)(i + 1) = I_(red)(i) − C_(extinction_red)^(′)I_(red)(i) = I_(red)(i) − αC_(extinction_red)I_(red)(i) = (1 − 0.3 α)I_(red)(i)

If the input opacity is 0, no energy is removed from the system, as maybe expected by the user.

If the input opacity is 100%, I_(red)(i+1)=(1−0.3α)I_(red)(i). At fullopacity, the extinction modulation model only removes 30% of the energyfrom the red channel.

Therefore, if the extinction modulation model were to be used alone, aglowing effect may be observed when the opacity that is input by theuser is high.

A light simulation may have a concept of energy, and of the preservationof energy. A system may be considered to be energy preserving if a sumof absorption and scattering equals the irradiance of a region. For eachregion, the incoming light is equal to the outgoing light plus lightabsorption. The concept of energy preservation may be applied to smalllocal regions and/or to a scene as a whole.

If the extinction modulation is used alone, the system in which it isused is not energy preserving in the extreme case of opacity=100%. Theuser may be surprised that setting opacity to 100% does not remove allenergy from the system.

This example of an approach to opacity has some properties which usersmay find confusing or undesirable.

We now consider what would happen if the transmission modulation modelalone were to be used to modify the transmission color, i.e. if only thetransmission modulation model were used across the full range of opacityvalues from 0% to 100% opacity.

In the transmission modulation model, the modulation circuitry modulatesthe transmission color by multiplying the transmission color by thetransparency.C″ _(transmission) =TC _(transmission)

where C_(transmission) is the transmission colour, T is thetransparency, and C″_(transmission) is the modified transmission colourwhich is modified using the transmission modulation model.

We consider again the example of C_(extinction)={0.3, 0.4, 0.7} andC_(transmission)={0.7, 0.6, 0.3}.

Using the transmission modulation model,

I_(red)(i + 1) = C_(transmission_red)^(″)I_(red)(i) = (1 − α)C_(transmission_red)I_(red)(i) = 0.7(1 − α)I_(red)(i)

If the input opacity is 100% (full opacity), I_(red)(i+1)=0. All of theintensity is removed, as may be expected by the user.

If the input opacity is 0, I_(red)(i+1)=0.7I_(red)(i).

Therefore, inputting zero opacity (full transparency) in thetransmission modulation model results in the removal of 30% of the redintensity. This would not be expected by the user.

It has been found that the use of the transmission modulation modelalone may create unexpected shadows in an image, which may be describedas rogue shadows. For example, in an image that is rendered using thetransmission modulation model alone, objects that should be fullytransparent may cast a shadow.

We have explained above that implementing a single master opacity usingeither the extinction modulation model alone or the transmissionmodulation model above may give undesired effects that are unexpected bythe user, particularly in extreme cases.

Another option for using a master opacity could be to use the masteropacity as an average channel opacity, for example using3α=kC _(extinction_red) kC _(extinction_green) C _(extinction_blue)

However, it has been found that an average channel opacity model mayproduce unexpected color changes. For example, the model in the equationabove has been found to change the blue balance, and is unable totransform full opacity into completely transparent. Even if opacity isset to 100%, the model still removes energy from the color channels.

By using the combined opacity model, transmission colors as selected bya user may correspond well to a final rendered output. For values ofopacity that are near 0, the first opacity model predominates, which isthe extinction modulation model. For values of opacity that are near100%, the second opacity model predominates, which is the transmissionmodulation model.

Unexpected special cases (for example, a material glowing when it issupposed to be fully opaque, or a material casting shadows when it issupposed to be fully transparent) may be reduced or eliminated. The useof the combined opacity model may be fast. It may require no costlyexponential functions.

Certain embodiments provide: a medical imaging method comprising: aphysical rendering method using wavelength/color dependent opticaldensity/absorption; and a master opacity ramp in which the optical modelis based on a smooth transition between an extinction modulation modeland a transmission modulation model from the points where these modelsproduce desirable optical properties.

The transition may be based on applying the extinction model at a lowopacity end of a spectrum and the transmission model at a high end ofthe spectrum. An analytical interpolation function is used tointerpolate between the models. An arbitrary function may be usedthrough a lookup table to interpolate between the models. A deepeningfactor may be added to increase/decrease the volumetric saturation inthe resulting optical model. A saturation UI dial may be attached toeach color control point or attached when the mouse is near a colorcontrol point. The deepening factor may be defined spatially and usedfor fusing a secondary volume. The secondary volume may be based on CTperfusion, PET, SPECT, Ultrasound Doppler, Ultrasound electrograph,simulated fluid dynamics.

Certain embodiments provide a medical image processing apparatus usingglobal illumination for rendering comprising: processing circuitryconfigured to: acquire a voxel value of volume data, memory, an opacitycurve which indicates a relationship between the voxel value and anopacity value, an extinction modulation model which indicates arelationship between a extinction value of the voxel and the opacityvalue, and a transmission modulation model which indicates arelationship between a transmission value of the voxel and the opacityvalue, set, the opacity value based on the voxel value and the opacitycurve, and an optical model corresponding to the voxel value, based onthe opacity value and based on at least one of the extinction modulationmodel and the transmission modulation model, rendering the volume databased on the optical model and the voxel value.

The processing circuitry further configured to: apply the extinctionmodulation model if the opacity value is closer to zero, and thetransmission modulation model if the opacity is closer to full.

The invention claimed is:
 1. A medical image processing apparatuscomprising: processing circuitry configured to: obtain volumetricmedical imaging data comprising a voxel value, obtain an opacity valuecorresponding to the voxel value, obtain an extinction color and/ortransmission color corresponding to the voxel value, modify theextinction color and/or transmission color using the opacity value,wherein modifying the extinction color and/or transmission color isperformed using a combined opacity model that combines a first opacitymodel and a second opacity model different from the first opacity model,such that there is a range of opacity values for which the modifyingcomprises applying both the first opacity model and the second opacitymodel in varying proportions, such that the first opacity model makes ahigher contribution to the modifying than the second opacity model at alower part of the range of opacity values in which both the firstopacity model and the second opacity model are applied, and such thatthe second opacity model makes a higher contribution to the modifyingthan the first opacity model at a higher part of the range of opacityvalues in which both the first opacity model and the second opacitymodel are applied, wherein the modifying is performed using a linearcombination of the first opacity model and the second opacity model forvalues of opacity greater than 0% and less than 100%, wherein the firstopacity model comprises an extinction modulation model and the secondopacity model comprises a transmission modulation model, the extinctionmodulation model being used only for an opacity value of 0%, and thetransmission modulation model being used only for an opacity value of100%, wherein modifying the extinction color using the extinctionmodulation model comprises multiplying the extinction color by theopacity value for the values of opacity greater than 0% and less than100%, and wherein modifying the transmission color using thetransmission modulation model comprises obtaining a transparency valuefrom the opacity value and multiplying the transmission color by thetransparency value for the values of opacity greater than 0% and lessthan 100%, and render the volumetric medical imaging data using themodified extinction color and/or transmission color.
 2. The apparatusaccording to claim 1, wherein the combined opacity model is such thatmodifying the transmission color and/or extinction color using anopacity value of 0% results in full transmission, and modifying thetransmission color and/or extinction color using an opacity value of100% results in full opacity.
 3. The apparatus according to claim 1,wherein the rendering comprises a color-dependent absorption process. 4.The apparatus according to claim 1, wherein the rendering comprisesglobal illumination rendering.
 5. The apparatus according to claim 1,wherein an interpolation function is used to interpolate between thefirst opacity model and the second opacity model over the range ofopacity values.
 6. The apparatus according to claim 5, wherein theinterpolation function comprises a linear function, such that thecombined opacity model transitions linearly from the first opacity modelto the second opacity model with increasing opacity.
 7. The apparatusaccording to claim 1, wherein the opacity value is obtained using anopacity curve which defines a relationship between voxel value andopacity value.
 8. The apparatus according to claim 1, wherein theextinction color and/or transmission color is obtained using a colormapping function which defines a relationship between voxel value andopacity value.
 9. The apparatus according to claim 1, wherein theobtaining of the opacity value, extinction color and/or transmissioncolor comprises receiving a user input comprising or representative ofthe opacity value, extinction color and/or transmission color.
 10. Theapparatus according to claim 1, wherein: the volumetric medical imagingdata comprises a plurality of voxels having different voxel values; andthe processing circuitry is configured to: obtain a respective opacityvalue and respective extinction color and/or transmission color for eachof the voxel values; modify the extinction colors and/or transmissioncolors using the combined opacity model; and render the image from thevolumetric medical imaging data using the modified extinction colorsand/or transmission colors.
 11. The apparatus according to claim 1,wherein the processing circuitry is further configured to obtain a valuefor a deepening factor that is representative of saturation, and whereinthe modifying of the extinction color and/or transmission color is independence on the value for the deepening factor.
 12. The apparatusaccording to claim 11, wherein the modifying of the transmission colorcomprises multiplying the transmission color by the value for thedeepening factor.
 13. The apparatus according to claim 11, wherein theprocessing circuitry is further comprised to determine a spatialvariation of the deepening factor based on additional volumetric data,and to apply the deepening factor to the volumetric imaging data set independence on the volumetric data.
 14. The apparatus according to claim13, wherein the additional volumetric data comprises image fusion data.15. A medical image rendering method comprising: obtaining volumetricmedical imaging data comprising a voxel value; obtaining an opacityvalue corresponding to the voxel value; obtaining an extinction colorand/or transmission color corresponding to the voxel value; modifyingthe extinction color and/or transmission color using the opacity value,wherein the modifying of the extinction color and/or transmission coloris performed using a combined opacity model that combines a firstopacity model and a second opacity model different from the firstopacity model, such that there is a range of opacity values for whichthe modifying comprises applying both the first opacity model and thesecond opacity model in varying proportions, such that the first opacitymodel predominates at a low part of the range of opacity values in whichboth the first opacity model and the second opacity model are applied,and such that the second opacity model predominates at a high part ofthe range of opacity values in which both the first opacity model andthe second opacity model are applied, wherein the modifying is performedusing a linear combination of the first opacity model and the secondopacity model for values of opacity greater than 0% and less than 100%,wherein the first opacity model comprises an extinction modulation modeland the second opacity model comprises a transmission modulation model,the extinction modulation model being used only for an opacity value of0%, and the transmission modulation model being used only for an opacityvalue of 100%, wherein modifying the extinction color using theextinction modulation model comprises multiplying the extinction colorby the opacity value for the values of opacity greater than 0% and lessthan 100%, and wherein modifying the transmission color using thetransmission modulation model comprises obtaining a transparency valuefrom the opacity value and multiplying the transmission color by thetransparency value for the values of opacity greater than 0% and lessthan 100%; and rendering the volumetric medical imaging data using themodified extinction color and/or transmission color.
 16. A medical imageprocessing apparatus comprising: processing circuitry configured to:obtain volumetric medical imaging data comprising a voxel value, obtainan extinction color and/or transmission color corresponding to the voxelvalue, obtain a value for a deepening factor that is representative ofsaturation, modify the extinction color and/or transmission color usingthe value for the deepening factor, wherein modifying the extinctioncolor and/or transmission color is further performed using a combinedopacity model that combines a first opacity model and a second opacitymodel different from the first opacity model, such that there is a rangeof opacity values for which the modifying comprises applying both thefirst opacity model and the second opacity model in varying proportions,such that the first opacity model makes a higher contribution to themodifying than the second opacity model at a lower part of the range ofopacity values in which both the first opacity model and the secondopacity model are applied, and such that the second opacity model makesa higher contribution to the modifying than the first opacity model at ahigher part of the range of opacity values in which both the firstopacity model and the second opacity model are applied, wherein themodifying is performed using a linear combination of the first opacitymodel and the second opacity model for values of opacity greater than 0%and less than 100%, wherein the first opacity model comprises anextinction modulation model and the second opacity model comprises atransmission modulation model, the extinction modulation model beingused only for an opacity value of 0%, and the transmission modulationmodel being used only for an opacity value of 100%, wherein modifyingthe extinction color using the extinction modulation model comprisesmultiplying the extinction color by the deepening factor for the valuesof opacity greater than 0% and less than 100%, and wherein modifying thetransmission color using the transmission modulation model comprisesobtaining a transparency value from the deepening factor and multiplyingthe transmission color by the transparency value for the values ofopacity greater than 0% and less than 100%, and render the volumetricmedical imaging data using the modified extinction color and/ortransmission color.